® 



fVUl. 
DOSSIER 



Europaisches Patentamt 
European Patent Office 
Office europeen des brevets 



lilillli 

© Publication number: 0 667 71 7 A2 



© 



EUROPEAN PATENT APPLICATION 



© Application number: 95300909.9 
© Date of filing 14.02,95 



© Int. CI. 6 : H04N 7/26 



© Priority: 14.02.94 JP 17276/94 

© Date of publication of application: 
16.08.95 Bulletin 95,33 

© Designated Contracting States: 
DE FR GB NL 

© Applicant: KABUSHIKI KAISHA TOSHIBA 
72, Horikawa-cho, 
Saiwai-ku 
Kawasaki-shi, 
Kanagawa-ken 210, 
Tokyo (JP) 



@ Inventor: Miyano, Yuichi, c/o Intellectual 
Property Div. 
Toshiba Corporation, 
1-1-1, Shibaura, 
Minato-ku 
Tokyo (JP) 

© Representative: O'Connell, David Christopher 
etal 

HASELTINE LAKE & CO. 
Hazlitt House 

28 Southampton Buildings 
Chancery Lane 
London WC2A 1AT (GB) 



© Method and apparatus for reproducing picture data. 



CM 
< 

rs 

CO 
CO 



© Apparatus for reproducing picture data includes 
a variable length decoder and Huffman decoder (22), 
an inverse discrete cosine transforming (IDCT) unit 
(23) coupled to the Huffman decoder, an intra-frame 
prediction unit and motion compensation unit (24) 
coupled to the IDCT unit, a memory (25) coupled to 
the intra-frame unit, a buffer memory (26) coupled to 
an output of the intra-frame unit, a resolution chang- 
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er (41) coupled to the buffer memory, a frames 
in/out unit (42) coupled to the output of the resolu- 
tion changer, a unit (43) for changing a timetable of 
picture frames coupled to the output of the frames 
in/out unit, a frame memory (44) coupled to the 
changing unit, and a selector (31) for selecting pic- 
ture data for reproduction from the buffer memory or 
the changing unit. 
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The present invention relates to a method of, 
and apparatus for, reproducing picture data. 

Figure 9(A) shows a known apparatus for re- 
producing picture data from a storage medium for 
displaying on a monitor. Figure 9(B) shows in more 
detail the decoder that expands and decodes com- 
pressed picture data, shown in Figure 9(A). As 
shown in Figures 9(A) and 9(B), a motion picture 
storage unit 11 includes a storage medium and a 
reproducing unit. A signal from motion picture stor- 
age unit 11 is input to a pre-processing unit 12. 
Pre-processing unit 12 amplifies and equalizes the 
signal from unit 1 1 . The signal from pre-processing 
unit 12 is provided to a demodulator 13. Demodula- 
tor 13 demodulates the signal from unit 12. The 
signal from demodulator 13 is provided to an error 
correction unit 14. The signal from demodulator 13 
is error corrected by error correction unit 14 in 
conjunction with a memory 15. The signal from 
error correction unit 14 is expanded and decoded 
by an expand/decode unit 16 to picture data in R, 
G, B format, or picture data in Y/Cb/Cr format, or 
picture data in CMYK format. The picture data is 
provided to a video signal encoder 17. The picture 
data from the expand/decode unit 16 is changed to 
a video signal (PAL or NTSC etc) in a video signal 
encoder 17. The video signal is then provided to a 
D/A converter 18, where it is converted to an ana- 
log video signal. The analog video signal from D/A 
converter 18 is provided to a display monitor 19. 
Display monitor 19 displays the video signal. 

Figure 9(B) shows in more detail the ex- 
pand/decode unit 16 in Figure 9(A). The com- 
pressed picture data received on input terminal 21 
is provided to a variable length decoder and Huff- 
man decoder 22. The compressed picture data is 
decoded in the variable length decoder and Huff- 
man decoder 22. The picture data from the variable 
length decoder and Huffman decoder 22 is applied 
to an inverse discrete cosine transforming unit and 
inverse quantizing unit 23. The picture data is 
inverse discrete cosine transformed and inverse 
quantized in the inverse discrete cosine transform- 
ing unit and inverse quantizing unit 23. The picture 
data from the inverse discrete cosine transforming 
unit and inverse quantizing unit 23 is provided to 
an intra frame prediction unit and motion com-: 
pensation unit 24. The picture data is predicted and 
compensated in conjunction with a memory 25 for 
intra frame prediction in the intra frame prediction 
unit and motion compensation unit 24. The picture 
data from the intra frame prediction unit and motion 
compensation unit 24 is output on an output termi- 
nal 27 through a buffer memory 26 for picture 
displaying. 

Variable length decoding and Huffman decod- 
ing restores compressed picture data by entropy 
coding. Inverse discrete cosine transforming re- 



stores compressed data by direct-alternating inter- 
change. Inverse quantizing restores quantized com- 
pressed data. Intra-frame coding is a method of 
forming an estimated picture by making use of the 

5 relationship between a picture and an adjoining 
block of the same frame. This coding is effective 
for compressing when a picture is simple. Inter- 
frame prediction coding is a method of forming an 
estimated picture by making use of a picture of an 

io adjoining frame. This coding is effective for high 
compression when a moving picture experiences 
only a small amount of movement. A method 
known as "H.261" of ITU (International Telecom- 
munication Union) can be employed to make use 

75 of estimated frames. A method of MPEG1 and 2 
(Moving Picture Experts Group Phasel and 
Phase2) allows use of bidirectionally predicted 
frames. 

As stated above, reproducing and inverse re- 
20 producing are often required. Conventially, a sim- 
ple method of inverse reproducing can be prac- 
ticed by providing memory between the ex- 
pand/decode unit 16 and the video signal encoder 
17. Some frames of picture data are stored in the 
25 memory, and the frames are reproduced in inverse 
frame order. In the case of the H.261 method, it is 
necessary to maintain picture data for at least a 
part of the refresh cycle. In the case of the MPEG 
methods, it is necessary to maintain picture data of 
30 one unit of a prescribed GOP (Group of Pictures). 
This is because it is not possible to reproduce 
picture data without using comparative picture data. 
Also it is necessary to change the order in which 
the picture data were stored. 
35 Accordingly, the invention is directed to a 

method of reproducing a picture and apparatus for 
reproducing picture data that substantially obviates 
one or more problems due to limitations and dis- 
advantages of the related art. 
40 To achieve the advantages of the invention and 

in accordance with the purpose of the invention, as 
embodied and broadly described below, the inven- 
tion is directed to an apparatus for reproducing 
picture data. The apparatus comprises means for 
45 decoding compressed picture data; a changer of 
resolution to change a resolution of the decoded 
picture data; a memory for storing the picture data 
with changed resolution; means for reproducing the 
picture data from the memory; and means for con- 
so trolling an order for reproducing the picture data 
from the memory. 

Further, in accordance with the present inven- 
tion there is provided apparatus for reproducing 
picture data. The apparatus comprises means for 
55 receiving compressed picture data compressed by 
inter-frame prediction coding; means for decoding 
the compressed picture data; a changer of resolu- 
tion for decreasing a size of the picture data by 
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decreasing resolution; means for omitting frames 
from the picture data from the changer of resolu- 
tion; means for changing an order of the picture 
data; and means for selecting the picture. data from 
one of the means for changing an order and the 
means for decoding, to reproduce the picture data. 

Also in accordance with the present invention, 
there is provided a method for reproducing picture 
data, comprising the steps of decoding com- 
pressed picture data; storing in memory the picture 
data with changed resolution; reproducing the pic- 
ture data from the memory; and controlling an 
order for reproducing the picture data from the 
memory. 

For a better understanding of the present in- 
vention, reference will now be made, by way of 
example, to the accompanying drawings, in which: 
Fig. 1 is a diagrammatic illustration of apparatus 
for performing picture data reproduction in ac- 
cordance with an embodiment of the invention. 
Fig. 2 is a diagrammatic illustration of apparatus 
for performing picture data reproduction in ac- 
cordance with another embodiment of the inven- 
tion. 

Fig. 3 is a diagrammatic illustration of apparatus 
for picture data reproduction in accordance with 
a further embodiment of the invention. 
Fig. 4 is a diagrammatic illustration of apparatus 
for performing picture data reproduction in ac- 
cordance with an additional embodiment of the 
invention. 

Fig. 5 is a diagrammatic illustration showing 

processing of compressed coded picture data 

according to an MPEG standard. 

Fig. 6(A) diagrammatically illustrates a memory 

map of memories shown in Fig. 1, and Fig. 6(B) 

diagrammatically illustrates a memory access 

map. 

Fig. 7(a) illustrates a memory map of a memory 
illustrated in Fig. 2, and Fig. 7(B) shows a mem- 
ory access map of the memory in Fig. 2. 
Fig. 8(A) shows a memory map of a memory 
illustrated in Figs. 3 and 4, and Fig. 8(B) shows 
a memory access map for the memory shown in 
Figs. 3 and 4. 

Fig. 9(A) shows apparatus for reproducing pic- 
ture data from a storage medium lor display, 
and Fig. 9(B) is a diagrammatic illustration of a 
decoder included in apparatus shown in Fig. 9- 
(A). 

Preferred embodiments of the invention will be 
described with reference to the drawings. 

Figure 1 shows an apparatus constructed in 
accordance with the invention and for carrying out 
a method of the invention. Compressed data in 
MPEG format are inputted to input terminal 21. The 
compressed data is provided to a variable length 
decoder and Huffman decoder 22. The com- 
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pressed data is decoded by the variable length 
decoder and Huffman decoder 22, and is provided 
to an inverse discrete cosine transforming unit and 
inverse quantizing unit 23. Data from the variable 

5 length decoder and Huffman decoder 22 is inverse 
discrete cosine transformed and inverse quantized 
in the inverse discrete cosine transforming unit and 
inverse quantizing unit 23. Data from the inverse 
discrete cosine transforming unit and inverse quan- 
go tizing unit 23 is provided to an intra frame predic- 
tion unit and motion compensation unit 24. The 
data is predicted and compensated by use of intra 
frame prediction in the intra frame prediction unit 
and motion compensation unit 24 in conjunction 

75 with a memory 25. The motion compensated data 
is supplied to a buffer memory 26. 

Data from the buffer memory 26 is provided to 
a resolution changer 41. Resolution changer 41 
changes the resolution of the picture data, and 

20 hence decreases the size of the picture data. 

The picture data from the intra frame prediction 
unit and motion compensation unit 24, after resolu- 
tion change in the resolution changer 41, is pro- 
vided to a frames in/out unit 42. In the frames 

25 in/out unit 42. frames of the picture data are omit- 
ted from and inserted in to the small sized picture 
data from the resolution changer 41 to increase 
and decrease the number of frames to control the 
speed of reverse reproducing. 

30 The order of the frames is decided by a picture 

frames timetable changing unit 43 and the frame 
memory 44. The frames from the picture frames 
timetable changing unit 43 are output on the output 
terminal 27 through a selector 31 . 

35 According to the present invention, since the 

resolution changer 41 changes the resolution of 
picture data, the amount of picture data to be 
stored in memory 44 is reduced in quantity during 
reproduction of pictures including changes in frame 

40 order and control of speed of reproduction. 

Figure 2 shows another apparatus constructed 
in accordance with the invention and for carrying 
out the method of the invention. In Figure 2 ele- 
ments which are the same as those in Figure 1 are 

45 identified by the same reference numerals. 

The apparatus in Figure 2 includes a single 
memory 51 which is used in place of the memory 
25 and frame memory 44 of Figure 1 . 

In reproduction of a picture, picture data pro- 

50 cessed by intra frame prediction unit and motion 
compensation unit 24, by use of memory 51, is 
outputted on output terminal 27 through selector 31 
and the picture frames timetable changing unit 43. 
At this time, the unit of changing of timetable of 

55 picture frames 43 does not change the order of 
frames. 

In reverse reproducing, picture data processed 
by the intra frame prediction unit and motion com- 

3 
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pensation unit 24, by use of memory 51, is pro- 
vided to the resolution changer 41. A picture size 
of the picture data is changed by the resolution 
changer 41. The picture data from the resolution 
changer 41 is provided to the frames in/out unit 42. 
In the frames in/out unit 42, frames of picture data 
are removed from or inserted into the small sized 
picture data from the resolution changer 41 to 
increase or decrease the number of frames to 
control the speed of reverse reproducing. 

The picture data from the frames in/out unit 42 
supplies an input to the picture frames timetable 
changing unit 43 through selector 31 . The order of 
the picture data is changed in the picture frames 
timetable changing unit 43 and the memory 51. 
The picture data from the picture frames timetable 
changing unit 43 is outputted from output terminal 
27. 

According to the second embodiment of the 
present invention, the apparatus for reproducing 
picture data can be connected with only the single 
memory 51. As compared with the prior art, the 
memory 51 needs a lower capacity when the frame 
order is changed during reproduction and the 
speed of reproduction is controlled. 

Figure 3 shows another apparatus constructed 
in accordance with the invention and for carrying 
out a method of the invention. In Figure 3 elements 
which are the same as those shown in Figure 1 
and Figure 2 are identified by the same numerals. 
In this case, data from the inverse discrete cosine 
transforming unit and the inverse quantizing unit 23 
is provided to one of the selector 31 and the 
resolution changer 41. Data from the resolution 
changer 41 is provided to another input of the 
selector 31. Data from the selector 31 is provided 
to the intra frame prediction unit and motion com- 
pensation unit 24. 

The picture data is decoded by the intra frame 
prediction unit and motion compensation unit 24 by 
a process in which memory 52 is used. The picture 
data from the intra frame prediction unit and motion 
compensation unit 24 is provided to the frames 
in/out unit 42. 

In the frames in/out unit 42, frames of the 
picture data are omitted or picture data is inserted. 
The processing by the frames in/out unit 42 utilizes 
capacity of the memory 52. The picture data from 
the frames in/out unit 42 is provided to the picture 
frames timetable changing unit 43. In the picture 
frames timetable changing unit 43, the order of 
frames of the picture data is changed, a process in 
which memory 52 is used, in order to reverse 
reproduce. The picture data from the picture 
frames timetable changing unit 43 is output to 
output terminal 27. 

During normal picture reproduction, the picture 
data from the inverse discrete cosine transforming 



unit and inverse quantizing unit 23 is output to 
output terminal 27 through selector 31 without 
omitting of frames from the picture data or chang- 
ing the order of the picture data. 

5 In the resolution changer 41 in Figure 3, the 

picture data is changed in each picture block of 
one frame. For example, in a frame subdivided into 
an 8 X 8 grid of 64 picture blocks, resolution 
changer 41 can reduce the amount of pixel data by 

70 deleting a part of the pixel data in each picture 
block. 

Figure 4 shows another apparatus constructed 
in accordance with the invention and for carrying 
out a method of the invention. In Figure 4 elements 

15 which are the same as those shown in Figure 1, 
Figure 2 and Figure 3 are identified by the same 
reference numerals. In this case, the picture data 
decoded by the variable length decoder and Huff- 
man decoder 22 is supplied to an inverse quan- 
go tizing unit 61. The picture data which is inverse 
quantized is provided to a resolution changer and 
an inverse discrete cosine transforming unit 62. 
The picture data is changed in resolution and in- 
verse quantized in the resolution changer and in- 

25 verse discrete cosine transforming unit 62. During 
normal picture reproduction, the picture data is 
only inverse discrete cosine transformed in the unit 
62 without changing a resolution of the picture. 
The picture data from the resolution changer 

30 and inverse discrete cosine transforming unit 62 is 
provided to the intra frame prediction unit and 
motion compensation unit 24. The picture data is 
decoded by the intra frame prediction unit and 
motion compensation unit 24, a process in which 

35 the memory 52 is used. The picture data from the 
intra frame prediction unit and motion compensa- 
tion unit 24 is provided to the frames in/out unit 42. 

In the frames in/out unit 42, frames of picture 
data are removed from and inserted into the small 

40 sized picture data to increase and decrease the 
number of frames to control the speed of reverse 
reproducing. The picture data from the frames 
in/out unit 42 is provided to the picture frames 
timetable changing unit 43. In the picture frames 

45 timetable changing unit 43, the order of frames of 
the picture data is changed, by use of memory 52, 
in order to inverse reproduce. The picture data 
from the picture frames timetable changing unit 43 
is outputted to the output terminal 27. 

so According to the present invention, the picture 

data is changed in resolution and is also inverse 
quantized. Changing of picture size in the inverse 
discrete cosine transforming step is equivalent to a 
filtering process, and thus it is not necessary to 

55 process by digital filtering after changing picture 
size. It is not necessary to have exclusive hardware 
for changing of picture size, and so it is effective to 
reduce the size of the circuit. For example, the 
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picture data that a low level of a coefficient of DCT 
(4 by 4 picture elements) selected from the picture 
data that inverse quantized as 1 block composed of 
8 by 8 picture elements. The 4 by 4 picture ele- 
ments are inverse discrete cosine transformed. It is 
possible to change picture size on account of pre- 
paring of brightness according to a rate of reduc- 
tion. This art of changing of picture size is de- 
scribed in "INTERFACE Jan 1993" in Japan incor- 
porated herein by reference. The art of MPEG1 
and 2 (Moving Pictures Experts Group Phasel and 
Phase2) is described in "INTERFACE Aug 1992" in 
Japan incorporated herein by reference. 

A further explanation regarding the capacity of 
memory and a form of using memory in Figures 1- 
4 is described next. Referring to Figure 5, there is 
shown how to decompress coding of each frame 
when reproducing compressed coding picture data 
according to an MPEG standard, wherein the I 
frame is an intra-frame coded picture, the P frame 
is predictive-picture, and the B frame is a bidirec- 
tionally predictive-picture. 

In reproduction, no reference picture is re- 
quired to reproduce the I frame. A P frame is 
reproduced from an I frame and P frame that 
precede it as reference picture. A B frame is repro- 
duced from two I frames, I and P frames, or two P 
frames that precede and follow the B frame as two 
reference pictures. Figure 5 shows in dashed lines 
the frames which are used to derive other frames. 

Accordingly, it is necessary to provide suffi- 
cient memory capacity for at least two frames of 
picture data to store reference picture data for 
decoding all frames. It is also necessary to have a 
buffer memory of about one frame for storing the 
balance of output picture data and controlling de- 
coding times. 

In view of the above, a memory capacity and a 
form of using memory is determined. Figure 6(A) 
shows a memory map of memory 25, frame mem- 
ory 44 and buffer memory 26 in Figure 1 . Figure 6- 
(B) shows a memory access map. This is an exam- 
ple in which only 1/4 of the normal memory capac- 
ity is required for each frame, on account of the 
change of resolution. The memory capacity re- 
quired to reproduce a frame is reduced to 1/4 the 
capacity that would be required if the resolution 
were not changed. 

Figure 7(A) shows a map of memory 51 when 
reproducing and special reproducing in Figure 2. 
Figure 7(B) shows a memory access map when 
reproducing and special reproducing in Figure 2. 
This is an example where the capacity of the 
output buffer is used as picture memory for some 
low resolution picture. 

Figure 8(A) shows a map of memory 52 when 
reproducing and special reproducing in Figure 3 
and Figure 4. Figure 8(B) shows a memory access 



map when reproducing and special reproducing in 
Figure 3 and Figure 4. This is an example in which 
the resolution of picture data is changed following 
the IDCT block 23. Memory 52 has the capacity to 

5 store many frames, for example 10 frames in Fig- 
ure 8, because the amount of memory required for 
decoding each frame is small. 

Thus, according to the present invention, it is 
possible to provide small scale hardware which 

10 nevertheless allows reverse reproduction. 

Claims 

1. An apparatus for reproducing picture data, 
75 comprising: 

means for decoding compressed picture 
data; 

means for changing a resolution of said 
decoded picture data; 
20 a memory for storing said picture data with 

changed resolution; 

means for reproducing said picture data 
from said memory; and 

means for controlling an order in which 
25 said picture data are recalled from said mem- 

ory for reproduction. 

2. An apparatus as claimed in claim *1, further 
comprising: 

30 means for receiving compressed picture 

data compressed by inter-frame/intra-frame 
prediction coding; 

means for omitting and inserting frames of 
said picture data from said changed resolution 
35 picture data; 

means for changing an order of said pic- 
ture data; and 

means for selecting for reproduction said 
picture data from one of said means for chang- 
40 ing an order and said means for decoding. 

3. An apparatus as claimed in claim 1, further 
comprising: 

means for receiving compressed picture 
45 data compressed by inter-frame/intra-frame 

prediction coding; 

a memory coupled to said means for de- 
coding; 

means for omitting and inserting frames of 
50 said picture data from said changed resolution 

picture data; 

means for selecting for reproduction said 
picture data from one of said means for omit- 
ting frames and said means for decoding; and 
55 means, coupled to said selecting means, 

for changing an order of said picture data by 
use of said memory to reverse reproduce said 
picture data. 
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4. An apparatus as claimed in claim 1, further 
comprising: 

means for receiving compressed picture 
data that is variable length encoded to es- 
timate in frames/between frames and discrete 5 
cosine transformed and quantized; 

means for inverse quantizing said com- 
pressed picture data; 

means for inverse discrete cosine trans- 
forming said compressed picture data; io 

wherein the means for decoding com- 
pressed picture data comprises means for vari- 
able length decoding said compressed picture 
data; and 

wherein the means for changing a resolu- 75 
tion of said decoded data comprises means, 
coupled to said inverse transforming means, 
for changing a resolution of said picture data 
by decreasing a size of said picture data to 
provide a lower resolution, further comprising: 20 

means for selecting said picture data from 
one of said means for changing a resolution 
and said inverse transforming means; 

means for decompressing data from said 
means for selecting to form frames; 25 

means for omitting and inserting frames of 
said picture data from said means for decoding 
in a reverse reproducing operation; 

means for changing an order of said pic- 
ture data from said means for omitting and 30 
inserting in the reverse reproducing operation; 
and 

a memory to temporarily store said picture 
data, coupled to said means for decompress- 
ing, said means for omitting and inserting and 35 
said means for changing. 

5. An apparatus as claimed in claim 1, compris- 
ing: 

means for receiving compressed picture 40 
data that is variable length encoded to es- 
timate in frames/between frames and discrete 
cosine transformed and quantized; 

means for variable length decoding said 
compressed picture data; 45 

means for inverse quantizing said com- 
pressed picture data; 

means, coupled to said means for inverse 
quantizing, for changing a resolution and in- 
verse discrete cosine transforming said picture 50 
data by deleting a portion of pixel data of said 
picture data; 

means for decompressing said picture 
data from said means for changing a resolution 
and inverse discrete cosine transforming to 55 
form frames; 

means for changing an order of the frames 
of said picture data from said means for de- 



compressing; and 

a memory to temporarily store said picture 
data coupled to said means for decompressing 
and said means for changing. 

6. A method for reproducing picture data, com- 
prising the steps of: 

decoding compressed picture data; 
changing a resolution of said decoded pic- 
ture data; 

storing in a memory the picture data with 
changed resolution; 

reproducing said picture data from said 
memory; and 

controlling an order for reproducing said 
picture data from said memory. 

7. A method for reproducing picture data as 
claimed in claim 6, further comprising the 
steps of: 

receiving compressed picture data com- 
pressed by inter-frame prediction coding; 

decreasing a size of said picture data by 
said step of changing the resolution; 

omitting frames from said picture data 
from said changer of resolution; 

changing an order of said picture data; and 

selecting from one of said picture data of 
changed order and said decoded picture data, 
to reproduce said picture data. 
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FIG.6 



Memory for decoding 
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Output Buffer 
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FO (low resolution : a part of 1 frame) 
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F2 (low resolution : a part of 1 frame) 



F3 (low resolution : a part of 1 frame) 
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FIG.7 

(A) 
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